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r . M'jl ti variate Mul tiple ' Regression 

in CoRimunicati on Research 

Far better an approximate ansv;er to the right 

question, which is often vague, than an exac t 

answer to the wrong question, which can always 

be made precise- . (Tukey, 1962, p. 13) ^ 

' The scientific study of human communication, like any other 
science, is fundamentally concerned with establishing lav/s of 
relations among the variables that constitute its subject matter- 
While formulating laws is, of course, a theoretical endeavor, 
establishing them is an empirical enterprise. When a communication 
scientist undertakes research to- establish a law, he mu.st choose 
a statistical techsn'que from among the -large repertoire available 
which in his judgment is the one best suited to" enable him to 
derive meaningful conclusions. This -paper will' present one 
alternative, multivariate multiple regression (MMR) and its uni- 
variate counterpart, which in my opinion is often- ideal ly suited 
♦:c this task. In fact,' as Blalock (1964) says, ''It is the 
regressi.on coefficients which give us the laws of science" (p. 51). 

It is not my intention, however, to argue for the superiority 
of MMR over other analytic techniques. Rather, it seems important 
'that the communication Eg,searchers understand the technique and its 
assumptions so that, as with all statis'tical procedures, he will 
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have a rational basis for selecting it v/hen it Is best suited to 
his needs. In Izhis regard, the assumptions are most critical for 
here is where the researcher must compare the nature and assumptions 
of the theory being tested (i.e., the proposed. law) with the 
assumptions or the analytic technique. Should the theoretical and 
analytic assumptions fail to correspond, then the 'analysis^ under 
one set of assumptions of data gathered under the other set is 
bound to be in error. 

The purpose of this paper, then, is to examine the multi- 
varidLe multiple regression model and to explore its applicability 
to the domain of communication inquiry, We^ will begin witri a prelude 
on partitioning a data matrix as a heuristic device^for distinguishing 
among alternative regression models. In the second section we will 
discuss, the major aspects of univariate multiple regression: (a) 
the form of the model, (b) using the model for descriptive purposes, 
> (c) estimators and their properties, (d) assumptions of the model, 
(e) procedures ■ for parameter estimation, (f) hypothesis testing 
procedures, (g) interpretation of regression coefficients, and (h) 
reestimation. The-third section will provide a parallel presentation 
for multivariate multiple regression. In the final section, we will 
turn to several additional topics. These will" include: (a) tests 
of assumptions, (b) coping with failed assumptions, (c) the relation 
of MMR to other oiu1tivariate procedures, and (d) th& advantages and 
disadvantages of MMR. ■ 
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Notation 

' In terms of notation it may be helpful at tr^. outset to nake the 
conventions we will utilize explicit. For parameters, i.e., 
characteristics of populations, we shall use upper and lower case 
Greek letters, e.g.,_ Z_, a", d_, p. For estiiiiators of parameters from 
sample data, we shall use the same Greek letters as the corresponding 
parameter augmented by a caret, "j- above each symbol, e.g., 6 as 
an estimator- of 3- For statistics, i.e., characteristics of 
samples, and for computational forms, we will use English equivalents 
(or alternatives, if necessary) to the Greek symbols, e.g. , ^ for Z^, 
b_ for 6^, etc. Vectors will be identified by lower case letters, 
matrices by upper case, and both will always be underscored to 
distinguish them from scalars and other data representations; thus, 
S is a single regression coefficient, B. is a vector of coefficients 
and-B^ is a matrix of coefficients. , 
Partitioning the Data Matrix for Regression 

Virtually every communication researcher has gathered data for , 
several, different variables on some number of subjects. The most 
traditional way to prepare these data for analysis is to arrange 
them into a subjects by variables data matrix. In fact, virtually 
alT standard computerized statistical packages require that data be 
prepared in this. way. Here the deck of punched cards may be 
considered the data matrix,^ where each card corresponds to the set 

• - 6 
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of observations (scores or measurerrients) on a single subject, i.e., 
a row of the data matrix. Since this format is generally familiar 
to researchers., let us examine various partitionings of this matrix 
in order to provide an overview of alternative linear regression 
models. 

A 

Suppose that we have a data matrix, ^^^^ , whi ch consists of 
measurements for N subjects (i = 1, 2, . . ., N) on M variables 
(j = 1, 2, . . M). Our matrix is then t order N x M. Let 
us assume that we are interested in how the values of some variables 
can be determined fr^r knowledge about the values of other variables 
regression analysis is appropr-iate for examining this kind of depend 
ency. Some variables in the matrix will te identified as predictor 
variables and others as criterion variables. We can partition the 
data matrix into two submatrices, one for predictor variables and 
one for criterion variables, and examine the interrelations among 
the two submatrices. 

According to convention, we will label the predictor variables 

•the X variables and the criterion variables the Y_ variables. We 

Y 

will label our criterion submatrix, , and our predictor submatri 
~F • It is possible to- have one or more of each kind' of variable, 

NxK 

i.e., one or more criterion variables and one or more predictor • 
variables, which provide four possible combinations or regression 
models. The first situation occurs If we have a data matrix with 
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only two variables. Then we 'would partition xrrr (^'' 
(where M = P + K = 1 + 1 - 2) 



6 - 

2) as follows 



Y > X 



NxP NxK 



NxT } Nxl 



fix 1 ' Nxl 
1 



With one criterion variable and one predictor variable we have the 
data partition for univariate simple regression. Note that since 

there: is only one criterion and one predictor variable, that the 

submatrices are really vectors (and is f'i x 2). 

Now assume that M is greater than two. We must decide how to 

partition the matrix. If we' choose one variable (the first, for 

convenience) as the criterion variable, so that P = 1 anu treat al 1 

the. remaining variables as predictor variables, {K>2), then we 



would partition 



NxM 



(M > 3) as follows 



(2) 



NxM 



Y t X 



NxP j NxK 



Nxl I NxK 



and cal.'i it the data partition for univariate multiple regressio n. 
There is a single vector of criterion scores but a matrix of two or 
more predictor scores (one score for each subject on each predictor 
variable). .. 



Now consider what would happen if we were to partition 



NxM 



so 



that there were two .or more criterion vav-iables (P > 2) but only 



one predictor variable (K = 1). 



. A 



NxM 



would look like 
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(3) 



NxM 



Y I 



N'xP 1 Mxj 



M = P 



P + 



I > 



3) 



This data partition can still be treated as a regression problem. 
Since there are several criterion variables but only one predictor 
variable it would be the appropriate data setup for a multivariate 
simple regression analysis. 

Finally, assume that the researcher is interested in examining, 
the regression of two or more criterion variables (p ^ 2) on two or 
more predictor variables (K > 2). Tlxen, 



(4) 



NxM 



^ixP j Nxk 



(M = P + k > 4) 



This partitioning of the data matrix into two submatrices is 

appropriate for analysis under the multivariate r;iult:ple regression 

model . ' ' 

During the remainder of this paper we will work with the 

Y X - 

criterion- and' predictor submatrices, and |^ » rather than the 
full data matrix. Since it is helpful in sorting out the differences 
for the various forms of analysis, the results of this section are 
summarized in Table 1. This data parti tioning .wi 1 1 also be useful 
when it comes to comparing the regression procedures with related 
techniques. 



Table 1 about here 
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While we will assuTie at the outset that the reader is familiar 
with standard sifTiple and nultiple regression techniques, a presen- 
tation of the.univariat2 inodel will help to set the staqe for cur 
discussion of the multivariate case. To that task we will now turn. 

Univariate Multiple Regression 
The Univariate Multiple Regression Model, 

The classical univariate linear multiple regression model can 
be given by • ^ • 

.(5) Y. = Bp ^ X.^ ^ . . . ^ &^ X.^ . E. 

whichshows the relationship between two or more (K) predictor or 

independent variables (X) and a criterion or dependent variable 

(Y) all measured simultaneously or. the subject. The model is called 

« 

linear (or linear in the parameters) because the effects of the 
various predictor variables are treated as additive, i.e., Y^. is 
composed of a linear combination of regression parameters. The 
regression parameters (Bq • ■ - are the population partial 
regression coefficients or weights which are determined from the 
sample data and used, to optimally predict Y.. Sq is a scaling 
constant which absorbs the differences in the scales used to . 
measure the Y and X variables. The e. represents the error term, 
i.e., the extent to Which the model fails to predict the criterion 



lU 



Multivariaie Multiple Regression 



scores, Y. 



'his can be seen by rewriting (5) as, 



(6) 



Y.. 



.Cv, - A., 



Since data are gathered on each subject (i = 1 . . . N) in 
the sample, there are N equations of ■:he form depicted in equation 
(5) above 



(7) 



h - ^'^h ' ^ '-21 ^ ^2 ^22 ' ■ 



• ■^2k + 



•2 



^k ^^k ■ 



These equations may be represented in matrix form as 



(8) 



or 



(9) 







^2 








y 

( 1 !x 1 





1 X^^ 
1 X^-j X22 



Clx.q) ■ (qxl) 



• ^Ik 




^0 




^1 






^1 


+ 


'2 


■ ■ ^Nk 




^k 







Jixl) 



'y is the Nxl vector of criterion or dependent scores, one for. each 
subject in the sample. ^ is the Nxq model matrix which contains an 
initial' vector of unifies and the data for the k predictor or 
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independent variables (q = 1 k). S is the q x 1 vector of partial 
regression coefficients and £ is the U x 1 vector of error terms. 

The model given in (9) is often called the raw or raw score form 
of the linear regression of or\ x^. Two other forrns of the fnodel 
are also possible and frequently encountered in regression work. 
Ore alternative following Tiinm (1975, p. 270), is called the 
reparameterized or deviation model . In this model all predictor 
scores are measured as deviations about their respective means. 
-Thus, the regression equation would be 

(10) y. = S, * r 6. I. * J B. (X, . - XJ * c. 
which can be represented in matrix notation as 

(11) Z - X. IL + £ 
Nxl Nxq qxl Nxl 

where q = k + 1, and is the model matrix of deviation scores, 
and n (i.e., etay is the vector of regression coefficients for the 
reparameterized model. 

The other alternative is to standardize the elements of both 
y_ and )(, which is accomplished by dividing each deviation score by 
the standard deviation of that variable. The standardized regression 
model is 

12 

o ' /- 

ERIC ^ 



(12) 



or 
(13) 
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1 — 






r 




. l2 




^11 


hi ■ ■ 


■ V 








^Iz 




hi 


7 • 

hi ■ ■ 


7 

• ^21 






f- 








hi 


hi ■ ■ 


•■ ^Nk 






























Z 




+ 








Nxl 




kxl 




Nxl 





note that the orders of the data matrix and the regression vector 
^j^^'in equation (13) are k rather than q = k + 1 - Since all variables 
are standardized to the same scale, there i-s no'longer the necessity 
for a- scaling term; hence, there is no initial vector of unities in 

X , . ■ . . ■ ' 

/ , and no Y in y^. 
Descr iption , . " 

' Suppose that a communication researcher has a set* of data which 
he wishe.s simply to describe , . Multiple-regression procedures may be. 
used^^ightforwardly in this case as a descriptive device. The... ' 
data will be divided into a criterion variable and one or more 
predictor variables. Using English lettp»^s .to. indicate that ■ ^ 
regression coefficients are;to be calculated from sample data for 
descriptive purposes only , ' the' regressi on equation specifying the 
relation among the ^ariables is given by 



(14) 



Nxl 



X - b + 
Nxq qxl 



e- 
Nxl 



- - ' Multivariate Multiple Regression 

where is an N .x 1 column vector of observations on the criterion 
spore (sometimes called the regressand), is the N x q (q = 1 ^* k) 
data matrix of observed predictor variables (sometimes ckl"! ' 
regressors) augmented by an initial vector of upities, b is le un- 
known q X 1 vector of regression coefficients and e^ is the unknown 
N X 1 vector of errors. The problem is one of determining the un- 
known regression coefficients and calculating the error components 
of the equation. . 

By manipulation of (14) we^cari create c,n equation which will 
give us a solutign for the regression parameters. : In order to 
obtain the best possible solution, we would- like our regression 
coefficients when multiplying the scores -to reproduce tha,^ 
scores as exactly as-possible. Another way to state this is that 
the error. component of the model, the difference between observed 
and calculated Y scores, or more precisely the error sum of squares 
will be at a minimum. First we calculate the error sums of squares 
e^e. ^ \ ■-. 

(15) e ,= y;- Xbv rearranging (14) 

* • e^"*e "= (y. - Xb)" (l - Xb ) multiplying 

= y."y. - yXb. bi'^'y^ + k"l"Xb collecting terms 
, = y."y. - VbXy. + b XXb . 
Since we are interested in finding the b^ v/hich gives the smallest 
error^ums of squares', let us differentiate these error sums of 
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squares with respect to b 



(16) - — = -2rz+2r- 

_ 9 b 

This solution-;to 9(e-'e)/3b, when set equ-. o; gives the "normal 
equations" , ' 

(17) rxb = Xx \ 

;Here,;X"X is the sums of squares and cross products for the predictors, 

is- the vector of regression coefficients, and X'y^ is the sum. of squares 
and' cross products for the/predictor and criterion variables. 

' Simple algebraic manipulation- of (17) giv^s us the solution for 
the regression coefficients we seek. > * , 

(18) ' : -b = (X^X)"'rx ^ 

For'those familiar with the ,calcul ati on of.simple. regression 'coefficients 
in summation notation it might be useful to point out that in the 
bivariate case (18) is equivalent to - 

.- ■ Vy>= = ^ ■ . . ■ ■ ' V 

which is . the ratio of variation between x and y to variation in x aione^. ^ 

Having now obtained ^ it is possible to insert the values of b 
into the equation to calculate .or predict , the values of y. Since we . 
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already possess the observed predictor scores, X_, the calculated 
scores , are 

{• ■ 

If we now compare our calculated'cri terion values, ^, with the observed 
criterion values, y^, we will discover the extent to which our pre- 
dictor variables! '_X, when multiplied by the best possible regression 
coefficients, b^, W^ajrately reproduce the observed scores, ^. As a 
measure of our fa i lure to correctly reproduce the observed score 
from our predictor variab':rS and the regression coefficients we 
construct an equation fc residuals. ^" 

/N ^ /\ . 

(21) e = V - Xb y. - since y = )Cb 

where e^ is the vector of residuals obtained by comparing, the observed • 
y^ scores with the calculated or predicted scores, y^^. 

Having obtained our best weights for the set of predictors ^ is 
useful to determine he well the prediction model, Xb_, "fits" th'i- 
observation vecto r y. ^- procedure, fol lov ing (^oldberger (1964, 
p. 159), is to parti ti or decompose the variance' of y_ into its 
component parts. The p..; :3 of the decomposition can then be developed 
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into a measure of the goodness of fit, R . First, we obtain the sums 
of squares for re siduals . 



(22) e-i = - Xb)' - Xb) 



= 11 - 2bX)i + bX/X (XX)'^ ry_ 



from (21) 
multiplying 

combining terms and 

from (18) 

since X'X (■XX)"V= I 



Also, from equation (20) we ,U! '^btain the suir^ of squares total 
for the calculated scores, v 



(23) 



^Tl = (Xb)^ (Xb) 

= b/rxb 



squaring (20) 
rearranging 
since Xb = y 



By substituting (23) in (22) 
of squares total for the obs-; 



^arranging terms, we obtain sums 
V scores. 



(24) 



This provides the fundamental Diit on for linr-ar regression.. The 
.parti tion states that total va -;- cn in observed^, can be decornpoFed 
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into two components: (1) sums of squares for predicted also called 
sums of squares for regression, SS,^, and (2) sums of squares error, 
SS^. If we treat the sums of squares as deviations from their 
respective means, then we have . •• 

{zsh ss^ = ss^ +. ss^ ■ ■ - 

••■ ' ■ ■ ■ . - ■■ . / " . ' 

"^ As a measure of the "goodness .of fit," the sample coefficient of ' 



multiple determination, R^, is calculated. 



,26) ■ , r2-. i .SS,. , .,_J31_-.^ 



which varies ■ between p and 1 . • As Goldberper (19M) s-ays , "When the 
fit is perfect, the least squares plane passes through every ob- 
served y, every e = 0, so"R .= 1. At the other extreme b-j = . . . 

= = 0, bp = y, the plane is horizontaV at y, every = y - y,. ^ 

9 2 
so R = 0." (p. 160). R, the square root of R , is called the 

sample multiple correlation coefficient. 

'The results of this section indicate that regres-sion analysis 

^may be used for purely descriptive purposes. Regression coefficients 

may be obtained and the" adequacy of the predictor variables may be 

determined as proportions of variance accounted for or as goodness 
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uf fit between observed and predicted scores. 

ConiniunxGation researchers are rarely interested in simple 
description. Typically, we draw samples and wish ta make inferences 
back to the population from which the sample came. In- regression . 
analysis, this inference will tyjically encompass two di f ferent but 
related processes: (1) estimation of population parameters 
(regression coeff icient^) ^'rom sample data, and (2) hypothesis tests; 
regardirq the parameters. • . 

Paramete r Estimation - . ^'^ 

The process-of makincj an inference about the value of a 
population parameter, 6, from a; sample , statistic is called estimation. 
An estiii ator, C, is a function or fo'-nula which tells how. to combine, 
'sample coservations in orcer to make tt" ^ estimate about the parameter 
An estiiv;ate is the value (scalar) obta ed for any given sample from 
an estimator formula. Accor"ding to Kni' ita (1971, p. 9) character-. 
•istics;of .estimators are derived by examing their sampling di.stributi 
The parameters typically 'estimated in regression analysis are the 
regression coefficients, . « .. 6,^, i .e. the" intercept and s.lo.pe 
values of the regression equation:' 

■ As many econometricians point out (e.g.^s^ Kmenta, 1971, . - 
Goldberger, ^964) it is iniportant that est'imators possess certain 
properties:^ - - ' . 



Multivariate. Klul tiple Regression 



18 



(1) An estimator should be unbiased. 6 is an unbiased 



<^stimator of e if E (e) = 9. This definition states 
that on the average, the estimator is correct, which 



:f the estimator equals the population oarameter. 

(2) Unbiased estimato-^s should be efficient. An 

9 is efficient if it has minimum variance among the 
:lass of unbias*:>d estimators. ^ 

(3) An estimator snould also be consi ::tent. This 
implies that as sample size gets larcrrr (i.e., 
approaches the oopjlation size), the estimator^ 
provides better estimates. An al ternai;' ve way to 
state this is zhat the "sampling distribLtion of the 
estimator tends to concentrate on the t ue value of 
the parameter, i.e., .it becomes less biased and 
smaller variance. . \ ^ 

(4) . Finally, an estinator that is consistent, is 

Best Asymptotic Normal .(BAN) . if the asymptotic 

distribution of /n \^^^-'^ :)) is normal with mean 0 and- ' 



nl ip*^ \t thr- me-^' 



the sampl ing distribution 



\ 



2 



(9) has the least possible value." 



variance a 
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Linear Unbiased Estimator (BLUE) (See, e.g., Goldberner, 1964, p. 1: 
Technically, bias, minimum variance and u.-ul ar^ \ m rred to as 
fU\. v^iiid i I ) sample properties, while consistency and BAN a.^e 
asymptotic (large) sampl properties. For an illuminating ar.. iogy 
betweer properties of estimators, and s^hootinc :t a bull's eye target 
with rifle, see;(Kmenta, 1971, pp. 13-14 and 168). Although it 
will i:.t be proved in this paperi^ it will come as.no surprise to 
many r ^ders, that the least squares procedures which were 
discus"^ ed earlier -or determining regression coefficients for 
descr. 3tive: purpose, turn out to be, under certain assumptions, 
BLUE of population parameters . We' now turn to those assumptions. 
A ssunutions of the Linear Regression ModeV 

In order co make parameter estimates on the basis of sample 
data it is necessary to make a number of ^assumptions about the 
populat'on. If these assumptions are warranted, then statistical 
theory regarding-sampl ing distributions and properties of 
. estimator's can be used tp formulate inferences about the parameters. 

The assumptions of the classical 1 inear regression model can 
' be summarized by several 'equations which will be'>briefly 'discussed 
in :his section. 

Assumption (1) is ^ - 

-» 

(27. = M + ^ . 
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v^hich specifies the functional form of the relationship in the 
population. It states that the observed scores, the , are 
linearly dependent upon the X^.^ scores and the disturbance or 
error terms, z^. 

Assumption (2) is 

(28) E (c) - 0 or, alternatively E(v) = X§ 

This assumption states that each disturbance term has an expected , 
value of zero. The two forms of the assi^mption are equivalent 
because if Z = M + £ and E(y_) = XB, then it must follow that ^ 
E(£) =0. 

Assumption (3) also may be written in two forms:' 

(29) V(g) = E(£^£) ^ oh or , V(i^) = a^i 

.The variance of £ equals ' the variance of because £ and y^ are ^ I 
separated on"^y by a constant,, which does not affect the variance. 
This equation states that the expected value for the sums of squares 
for the disturbances will equal a constant variance times the 
identity matrix. This' really encompasses two assumptions. First, 
assumption (3) specifies that for all values; l^y of any given 
predictor variable, X., the variances will be constant, i.e., they 

■ 22 .. ■ 
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I ..will be hom oskecastic . The assumption of homoskedasticity is some- 
times written as 

(29a) E(e?)-= ^ . 

2 Z 

Violation of the assumption; i.e., E(e^. ) ^ a is referred to as 
heteroskedasticity. Second, since an identity matrix has zeroes, 
in the off-diagonal .position, (3) implies that the errors are 
• • uncorrelated, which can al.so be written as 

(29b) E(e.e.) = 0 for alT i 

This assumption is often referred to as one of the nonautoregression 
or nonautoregressive disturbances. 
■ Assumption (4) stipulates that 

' (30) ^" X is an N X q matrix Which is fixe(d in repeated samples. 

This assumption implies a nonstochastic X which further implies that 
X^and G are independent. | . 
Assumption (5) is that the 

" (31 ) Rank of X^ =^ q 1 I 

EKLC 
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which indicates that there are more subjects than variables and that 
no exact linear combinations exist an^ong the predictor variables. 
This latter statement, that there is no exact correlation among the 
predictors, is often referrea to as lack of multicollinearity. In 
practice we are often more concerned with high degrees of multi- 
collinearity than with perfect multicollinearity. 

If we add one additional assumption, (6) that the e. are 
normally distributed, to the previous five, the model , becomes the 
classical normal linear regression model . Assumptions 1 - 3 and 
6 may be compactly . summarized for this model as 

(32) ^ 'x. N(X6, a^i) or , alternatively e v N(0,Z) , where* 

♦ 

This format for synthesizing assumptions 1 - 3..'and normality may be 
read as "Y is a normally distributed random variable with 
expectation (mean) equal to X6, and variance equal to a ^, or^ 
alternatively, "The errors are normally distributed with zero mean, 
and equal variance, E^. ■ It is also often assumed that the X:. 
scores are measured on scales that have at least ordinal properties, 

but that is not required by either model. 

• ■ .-* *-■ 

Estimating Univariate Regression Parameters , / < 

Having stated a model of the form 
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and specified a number of assumptions about that model, how can we 
obtain BLUE estimates of the regression parameters? If the assumptions 
are valid, it turns out that the least squares procedures that were 



utilized for descriptive purposes, may also be used for inferential 
purposes. Using the carot (") to indicate that sample data are 
being used to estimate the ■ population regression parameters, 
equation (9) is rewritten as 



(33) y , X i I 
Nxl Nxq qxl Nxl 



- yihere the details of the vectors and matrices are as previously 
specified. Equation (33) -may' be manipulated- in' identical fashion 
to the way equation (18) was derived from equation (14). These 

operations lead to the least squares estimate of the population 

' ' ^ " ' ' , - \ • 

regression, coeffici eats,. (See Finn, 1974, p. 97, for a demonstration 

. tf* ' 

' - *N , , 

- without using calculus that B -is a minimum sum of squared errors 

; estimate of 6 • ) • ' . 



(34) '■ i= (XX)'\-X ^ 



3 is ah unbiased estimator of 6_ because E(B) = B . .. This can be shown 
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fairly easily. 



(35) 



(rx)"' ri 



= (rx)'' X E ()^.) 

= (X'X)'^ X'XB 



= B 



taking expertations of (34) 
since X is constant 
from Assumption- 2 

E(Z) = M , . - 

since (X-'X)"^^ X'X = 1 ' 



B is also efficient, i.e., minimum variance. [See Finn, 1974;, 

p. 99, for proof). Arid .in general, 3,is a BLUE of B'(see Kme.nta,. - 

1971 , pp. 209-216). - . ■ 

Since an estimate, though unbiased, will be correct only on 
the average (i.e.,. E(0-) ^ 6), it is^ important to obtain an estimate 
of the variability of B over repeated samples (of the same size). 
Th.is, of course, is equivalent to asking, ''What is the variance or 
standard deviation of the- sampling distribution of the estimator, 
in this case 3^?" The. variance of'j3 (see Johnston, 1972, pp. 125V 
126) is given by 



(36) 



V(i) = a^'diag (rX) 



-1 



■and the standard deviation of 3» called the standard error of 
estimate of 8. is giv^en by * . • 
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(37) - 0(6)' = a diag (X X)'^ 



Thus, for the >i-^regression coefficient, B:;^ the standard error of 
estimate is 



(33) . \ = a diag (X X)j / 



Both the variance of g. and the standard error of estimate can be 
found on the diagonals of their respective matrices. 

The conventions regarding .reporting of regression results in 
economics might usefully be. incorpoV'ated into cormunication research. 
As Kmenta, 1971, indicates , "It has ^2come * customary to present all 
these, results by writing out the estimated regression equation with 
the estimated standard errors inf parentheses under the- respective 
coefficients. This is followed by the value of R " (p. 242), For 
our development we would write 



(39) Y. = . P'^ X. . - . X, . e. ■ ,2 

(%)("Bi) - . . ' (\) ■ 



The foregoing indicates^haj. the procedures for determining^ 
regression coefficients for inferential pur '^ter estimation) 

are no different from, those employed for descrip.; jrpos^:.. What 
does differ is that in the former case a number of critical 
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assumptions are made about the population, while in the latter case, 

no assumptions are made. These assumptions, .of course, make all 
the difference. Depending on the extent to which they are invalid, 
the researcher must either abandon ordinary 'east squares "^(OLS ) ^ - 
techniques in favor of other alternatives (e g.:, two stage leas.t 
squares, instrumental variables, etc.) or be left in the unenviable 
position of being able only to describe relaions in the obtained 
data." He will briefly review those alternatives at a later-point 
in the paper. Now, having shown. how to obtain parameter estimates, 
we will brief ly review various interpretations of regression co- 
efficients before turning to the important topic of how =to. test 
the significance of those estimates.- . ' 

Interpretation of Regression Coefficients 

It might be helpful at this point to be explicit about the 
interpretation of the various regression coefficients. As already 
mentioned, the parameter 6^, its estimator counterpart, B^, and the 
.descriptive sample coefficient, b^. are all scaling coefficients 
whi^h are necessitated if ^ and are measured on different scales, 
i.e., in different units of measurement. ' These coefficien.ts 'insure 

the equality of the right hand and left hand sides of their 

■* ' • i 
respective equations. v ^ 

(40) y " p^o ^ ^1^ ^ ' • ^ ^ -/\ " ' 

■ '"v. \ - y - (B^x^ + . . . ' 
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.re same scale, the coefficient 
ized model where all variables 
ore transformations there is 
Y is the standardized 

' 0 

/\ . ■ ,, 

B and b are defined only 
0 0 . 

regression model. Each may 

line, plane or hyperplane 
.3on whether it is a simple 
ession, or greater than two 
. As an intercept, it is the 
'ctor case, when and X^ = 0 
(|^ = 0 in* the greater than two 

pointed out that many texts 



n . . v k' ^1 . . . 

on coefficients. They should 

unit change in X . . This 
the slope of the line', plane 



, , and 



or hyperplane of the regression of y on Xj. Alternatively, they 
may be thought of as weights which are used as multipliers for the 
X. in ^.rder to optimally predict ^y. They are called partial 
regression coefficients because each shows the re.lationship between 
y and a given X- partialing out or controlling for -the influences 
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only exceotion .is is che intu 
;ei , p , which is why i; ome authors prefer 

G , , 
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social sciences is to call tne s tandarcized raqression cdefficier 
a "beta" weight arid ic reserve for it the symbol "g." This is i2^_ 
the convention in ecor..:nietri cs and many multivariate texts where 
Greek letters are reserved for population parameters or estimators 
(Finn, 1974, in fact, reverses the convention and uses 6 for the 
unstandardized anc b for the standardized coefficients. Gold- 
berger's, 1964, usage is such that he directly warns his readers 
about not confusing "beta" coefficients with the elements of the 
population coefficient vector Sorting out the variety of 

conventions of b and 3 is not always easy and it is the main 
reason why, following Timm (1975) and others, I have chosen to .u? 
an, entirely different symbol, y, for the standardized coefficier 
The important thing to remember i.s not the syrr.bol ; rather, remeni: . 
(1) there are ty/o forms of the slope coefficients, standardized 
unstandardized, (2) the interpretation appropriate to each, and 
(3) that each may be defined as a parameter an estimator or a 
descriptive statistic. 

Hy pothesis Testing • . . - 

Hypot esis testing ir regression analysis is the procedure 
whereby inrluences about che significance of population regression 
-coefficient:; (parameters) are' made on the basis of sample regressi 
coef^f icients (statistics). As Finn (1975.., p. 134) indicates", this 
process requires two separate steps: {]) partitioning of variafio 
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cc" - ' • 3 for 
con:: ■ : . f 

ar f "'"01 or 
and ' tomparisc- 
::: : squares 



tr,-: to- 



>r:ance, SS^, 



■ nto sums of sc.ares for regression, SS^^, and th .:e ''jr error, SSj- 
Recalling equal" ons (22 - 25), ' et us now use ■ cin estimator 0^ 
c_ (instGc J of 'j::inc -^or cescr'otive purpose^) d id call this - 
scalar qi .ntity. fc • conven'ence, Qg. ""tius, the of squares 

(and crc:t products due tc error are 



(42) 



fron (22) 



an. the sums of sc ^es (enc cross pr iucts ) hy;: t tri^s i ^ , or as it 
is o~ten called, of sq; :^res for rgression -ro 



(^3) 



^or: 



We can how estab" 
eratle us to, test 
jo itly signi f ica: 



nil*" " alternat -^e •'cvr 
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that wil ' 
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V.' tne a "i i:- . ti ve 

To cest ,-1 we ' =^1 cul i-r.^ a standard ra .r of the t /c • 
of squares d-'v;ded b: the " :.ec-ees of freedor '.e., a ratio 
twc irean Si .. "es. Mi and 'm^:.,. . 



An ^'.OVr table ■:ab]c ' ,may be co^' ' ^uc:: -:^c ::o sufnr-^r^ize the 

"MK 'jrmation ^.'^r te:t :h k-1 ) n-' ".eqrees :f free:.-:' 

TIvs p^^ '-r—L.^;'- t^s::, :nou?h the \ie-^cl, is rr- little 

. 'tica'I '.. ^it ircp sts t' :igre:::sion c:):efficien 

■nc]L:iinc en- '"'•nt :,. . "o ^'Ve ■ ' : preblem, ^ is 

osi '^le .mir- ^e eqi. ivele' ' ' ; r ne etandard'zed vec . 

f r^gres ioi' ;:/e'^- . ^ ' i tne veceer into 

wo : ibsr'.s, . -r^d e^^Pii-* ly the sloee coeff 

— ' 0 

see Timr, 1975. -; anc' fiolcberne!' :964, pp. ""'5-177' 
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v/hich hypothesize that the va-^iables ir tr-e second set w^ni :h 

correspond to 3, significantly contribut'i tc the variance y_ over 

n 

and above the regressors in the first set, 

. ' A convenient technique for testiuiq z\v.s hypothesis is to 

formulate two separate regress'ion equa*:icns. The first ecjation 

contains the first m of the k predictc-": (t. <k) which ar- know^n or' 

assumed to significantly contribute i:^ :^"'terion variar/::,-; the 

second equation contains all predictcr-^s , Le., the firs: ni 

variables not being tested :)1 js the reina' n; nq ^. - m variables 

which constitute B. . The resoecti've equations would be 
-Hi 



(50a) 3 : = X B 

' Nxl Nx(ni+n (-1+1 )xl 



anc 



(50b) : y: = X_ i '\- e where c - k 4- 

, Nxl Nxc :\].'\ Nxl 



These two equations can, of coi rse/ be reprr^sented by theic 
respective measures of "goodnes? of fit." for th: re:ii ced set of 
regressors, B^, and R^^ for t;- - so o^^ regresso* - . ^ ■ = : . 
It is then possible to cons , l F :.is:J^ that M -"^^able 
to determine the significan: ■ r le B^. v- ..ij)- ^nd t'e 

B vector. 

'22 ' 
P - P 

(51 ) F - h \v N - k - I 
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which hc5 (*' - ir and - k -1) degrees of freedom, m equal to 
the number cf X variables in the reduced vector B^, and k equal to 
the full se-: of X '/ar' ab^ : in S.. If £ is significant at P < a, then 

is rejected end it is cncluded that s ome of the regressors in 6^ 
do contribute stigini ficant y to variation in ^. The squared partial 
multiple coe^^^ici'Bnt o"" letermi nation is given by 



• 2 2 

(52) K , - \- - .^m 



[fl ^ k) • (1, . . .,'m)' 



m 



and indica the amount of variation in Y which remains after the ■ 
first m ya Fables have been partialed out of the regression equation. 

When ^ . cont-:: ns only one regression coefficient^ so that = k - 1 
(i.e., 3^ c.ntain- all but the last^coeff icient) , equation (51) has 
a special i iterpr :ation which is of considerable importance. 
Specifically, the equation can be shown to reduce to the partial 
co rrelati::n coeffic ient (see Timm, '1975, p. 277) 



1 - 



v;h'ich when examined as an F_ test at 1 and N - k 7 1 degrees of freedom 
shows whether one^ X variable added to an existing set of predictor 
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variables contributes sianif icantly to variation in the criterion 

measure. Thus, we can test whether an individual regression 

coefficient differs significantly from zero, and thereby contributes 

2 

significantly to criterion variation. The size of r^^ ^ 

k - 1) also provides a measure of the shared variation between the 
two variables. 

There are several other hypotheses that are often worthy of 
.examination; they are listed in Table 3 along with those that have, 
already been discussed. 

The formula provided in equation (53) is perfectly adequate for 
examining the contribution to cri terion variance made by the last 
variable in the equation: ' In general, however, if we examine more 
than the final variable we find that the regression coefficients for 
pairs of predictors are generally correlated. Let us. suppose that 
we wish to obtain individual sequential tests for each of the final 
-K - m>l predictors,, i .e. , the last variable, the next to last 
variable, etc,,* until we get back to the variables. Finn ( 1974) 
indicates that 

A series of independent tests if facilitated by 
transforming the predictor variables to a new set of 
uncorrelated measures, in a specified order. We 
shall substitute for predictor Xj in only the linear 
function or portion of X- that is uncorrelated with 

Table 3 about here 
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preceding predictors X^, X.^, . . . X^. ^ That is, 
we shall find the Xj values that are obtained if we 
"partial out" or "hold constant" the effects of 
earlier predictors in the set (p. 137). 
There are two important points to note here. The fi^^st is that each 
variable is orthogonal ized (made to be uncor^rel ated) with only those 
variables that preceed it in the equation. The second is that 
the process requires that the variables be arranged in a prespecified 
order. 

The process which Finn (1974, pp. 134-144;. see also Timm, 
1975, Kshirsaga'r, 1959, and Anderson, 1958) describes produces 
orthogonal estimates of regression coefficients or' semipartial 
regression ' coefficients . Each semipartial regression coefficient 
indicates the relation between Y and X^ controlling for the variance 
contributed by variables preceding X. in the equation, but hot 
controlling for the variance of those variables which follow it. 
The process "of sequentially estimating the effect of each predictor 
variable eliminating those preceding it, is called stepwise 
el imi nation . 

2 

In general, the r s between Y and each X^. will not add up to 

? 2 • . 

Ry . because the r s are correlated. The squared semi- 

1 . . k ' 2 

partial correlation coefficients will, however, add to R ^ 

2 ^ ^ • • • 



since the constraint i'S imposed that each sequential r^ 
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as Kerlinger anc Pedhazur (1973, 94) iraicate 



'/ariables. Thu 




r 



yi^. ) 



3 n...2 



k '^^..^c-l 



which is r-rad, "The total anount f shared variance in y and X_ 

is given by the sum of the squa ;i semiijartial correlation 

coefficients, i.e., the squared rrelation between Y and plus 

the variance bet^eer Y and remioving the influence of from 

the re~ation wich Y, plus . . . p^'lus the variance between Y and X, 

k 

removing the influence of X^ through X|^_^^ from the relation with Y." 

Often a researcher is interested in whether the combined effects 
of two [or more) variables viewed together significantly contribute 
to criterion variation over and above the variation accounted for 
by the variables viev/ed separately. This interaction between (among) 
variables can be examined in regression analysis by the inclusion of 
multiplicative cr :ross-product terms in the predfiction equation. 
Each intei-action term is treated as a new predictor variable in the. 
linear model. The new variable is created by cross multiplying the 
observed values for each subject on the original variables, i.e., 

'^ij/ij2' ^^"^^^ ^^^^^ linear this multiplicative term 

is added to the model and a regression parameter is estimated for 
the new term. Ffor example, given two variables, X, and X^, the 
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linear model which includes the interaction term would be 

\ 

(55) y. - 6^ + 3^X.^ ^ ^^y..^ +• 63X.^X.2 + 

To calculate interaction terms it'fs'first necessary to 
standardize 'tVe respective variables prior to computation of the 
cross-product terms. As Finn (1975) ind^icates, "the dominance tff 
the interaction by one or another variable due to .scaling is 
avoided. The interaction terms themselves need not be standardized" 
(p. 85). Once this is accomplished, the .OLS estimation and 
hypothesis testing techniques described earl ier can be 'employed. ^ 
If should be noted, however, that it is customarjr to add the 
multiplicative terms into the equation last, t|us testing them 
first if backward elimination techniques are employed. 

Several hypotheses have been identified in 'this section which • 
a researcher might prof itably examine. These do not exhaust the 
possibilities^ however. ,Table 3 presents those discussed plus 
several other alternatives which are worthy of consideration. 'Which 
test to use depends, af course, upon ^ the theory quiding the research 
Perhaps an example will help to il lustrate this p'dint. 

Attentibn in comm.unica'ti'on research Uends to focus almost- 
exclusively on. slope regression coefficients, i .e. ,3^. (i =1 • • • k)' 
to the exclusion of the intercept coefficient, 3^{or a). Unless 
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data have been reparameteri zed to force the intercept to the ' 
origin (or some other level) of the coordinate system, the value. of 
the intercept may also be of considerable theoretical "interest^. If 
we modify an example given by Kmenta (1971, pp. 204-205) we cai^ see 
the importance of the intercept. Suppose that media consumptior| 
(i.e. , number of hours/week spent watching. T.V) , Y, is regressed on 
amount of leisure time (number _ of hours not devoted to gainful 
employment or primary occupation), X, and. a linear relationship of 
the form E(Y) = 3^ + B^X^ + e is determined. The slope coefficient, 
B, would be interpreted as the marginal propensity to watch T.V., 
i.e., the amount of increase in 'watching T.V. for every unit C 
increase in amount of leisure time. Now examine B^. Assuming that 
people will consume at least some T.V. even if they have no leisure- 
time, B^ indicates the minimal or subsistence consumption of T.V., 
i.e., the amount of T.V. exposure when the value of leisure time 
is zero.'- Hypotheses regarding the intercept coefficient could be 
of significant theoretical value. Refer ing to Table 3 we might 
Utilize test (1), that H : 3. 0, for the intercept coefficient 

O K [ 

6 If' our theory were more sophisticated vie might utilize test 

0 • 

(2), that : Bq = Mj^, for some constant, which is tantamount to' ^ 
asserting that we can predict /'subsistence" media consumption at 
some number greater than zero. - 
Reestimation 

As was pointed out earlier in the pape"r, estimates of 
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regression parameters are valid, under the assumptions of the 
model, only for a given set of regressors. If we change the set 
of regressors, either by adding or deleting predictor variables, 
the coefficients for the original equation for which the estimates 
were obtained will no longer be appropriate. New estimates should 
be* ot2j:^'ined. - - - 

r It should be -clear that the hypothesis testing procedures 
drscnbed rn the previous section lead , if significant, to the 

""deletion of( variables from the full set of variables included in 
the original equation. Or, alternatively conceived, the procedures 
lead to the addition of significant variables over and above those 
already included in the equation, (it is possible, of course, to 
test the entire set, of variables.) Under either conceptualization, 
obtaining significant results in hypothesis testing is likely to 
lead to a different set of variables, a different regression 
equation, than the one with which the researcher began. 

In situations where this ■ occurs, the researcher should 
reestimate the parameters of the final obtained regression equation. 
The procedures for reestimation are identical to those for 
estimation. The final set of variables are estimated by OLS 
procedures, with the variance of the rejected variables pooled 
with" the error variance. A final estimate of the standard error 
of estimate should" also be obtained. The final equation which 
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contains the final estimates -of regression parameters, the final 
standard errors of estimate, and the recomputed final R should be 
the results that are presented in the research report. 

Multlvar :ite/Mul ti pi e Regression 

In this section we w t1 extend the findings developed for the ^ 
univariate model to the inu] ti variate case. Most of the' presentation 
will be by analogy with univariate results. c 
The Multivariate Multiple -egression Model ■ - . 

Let us assume that a :cmmunication researcher has two or more 
variables that he would 1 examine as criterion variables in 

relation to the same set .^pfredictor variables that we assumed in 
the univariate case. Th is a problem for which multivariate 
regression^ analysis is appropriate. 

The multivariate multiple regressioh model .is given by 

(56) Y ^ X ' B ^ E . 

Mxp Nxq qxp Nxp , - 

This equation states .that a data matrix has been partitioned into 
Y 

two submatrices which contains two or more criterion, variables 

and which contains all the data for the predictor variables 
Nxq 

augmented by an initial column of unities. Furthermore, the scores 
in the Y_ matrix are composed of linear combinations of the X scores, 
each weighted by^a regression coefficient, |3|^\ and added to' the. 
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error term, c .. That is. 



E-, then, is an Nxp matrix of error terms, with one column of errors 
for each criterion variable. If the model is intended to be a 
multivariate classical linear r-:;; }ssion model we assume 



(58) E(Y) = XB which - 

since the expectati on 'of Y is 



e^uivale-t. to E(£) = 0 

.id 



(59) . V(Y) = S I w^iMcl 
si: e and i differ only t^y'va 
,.pr; uct oper:;tor, , v/hich"is 
produces rn :his case a diagona 
equal to ^ tind off-diagonal V 



(60) . V(Y) = V(E) = I^j ® 



luivalent to V(E_) ® 
' t, nanit: ' X!B. The 'Kroneci- er 
J define the variance of Y, 

"ix with diagonal elements 
0. ^hus 



0 . . 0 
0 ... 0 

0- 0 ... Z 



These assumptions may be sun^iiarized as 



(61) 



E ^ N,, (p, I® E) 
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which states that Y is- a multivariate normal distribution, that the 
errors have constant variance, and that the errors are uncorrelated. 
'it is also possible to state the model in equivalent mean deviation 
(reparameterized) and standardized ■ forms . In the former case - 

(62) 1 ^ + ^- 

Nxp [^x- qxp ^xp 

where eta (H) is the matrix of partial regression coeff icien-:: 
the mean-adjusted scores, X^. In the latter case we have 

(63-) . Z r- ^ E •. 

flxp. rixk k,\.: Nxp • 

where gamma (r^) is the matrix of standardized partial -egre^ 
coefficients. , • - ■ 

. Estimation of Multivariate Regression Parameters 

In order to estimate the regression parameters we procee b: 
in the univariate case. The process is to apply the Gauss-Mar -.off 
Theorem ("see Timm, 1975, 185-188) to obtain the matrix B so that 
the sum. of . squared err6rs,is minimized. The theorem states that 
this can be accomplished by minimizing the trace of the sums of 
squared errors, matrix, £%■ Thus, let equal to the sum of 
Squared errors to be minimized, then - 
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■ • / , ^ ' ^ k p .-, 

(64) / = Tr (E'E) " since Tr (E;E)= v >] t 

' ' ■ ■ ' " , . ^ 1 = 1 j = 1 ■ 

... = TR . (Y - XB) ' (Y - XB) rearranging -(56) 

When the partial de)"^ i vati ves of with respect to are set- equc 
to zero and solved, the following normal equations are obtained 

(65) ■ ^ , " IT 
>^ ch'Ccn be solved for 

(56) B y XT 

This equation states that under the assr'nptior of :ne model the best 
linear unbiased estimates of the matrix of ve'~ ^ession coefficients 
can be obtained directly from. the sums of squares and cross products 
matrices of the raw data, i.e., from X^T the inverse of X_T- 
Comparison of equation (6f) with equations (18) and (34) indicate^: 
that the multivariate solution is completely analagous to its 
univariate counterpart. 

B^, of course, consists of a matrix of partial regression \ 

r 

coefficients. For each criterion variable- there are as many 
coefficients as there'are predictor variables, plus one- coeff i cient 
for the intercept. Thus, if there are two criterion variables and 
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three predictor variables, there would be 2 .x + 1) = E regression 
coefficients. In ge'^eral, us^'ng the orders of cv matrices, there 
will be (q x p) regr r>sion coefficients (rerefroe , c k predictcr 
variables 1) for tr^e raw forr of t'le rer^-rssic model. In the 
St- rdardized :nor:fl : "'^re would be x ^ coef f: ::ients. 

'Since _B has beer estimate: frorr the r'ata, i": is desirable to 
obtain a measur- ' ~' :::\e stabi^'^y of . ::imate. Sampling theory 
specifies tnat reared sampl inc or a given population we 

would obtain a - "i. Jticn of Bs fo ^mples of a given-size, N. 
"hat sampl" v: d 3t .: ;ion would ha\ oth an expected value and a 
ariance (as we l as ^ ghe^' order mome s). Finn (1973) indicates 
lat "The estir ce b^, like its Univarie ;e counterpart, is unbiased 
id minimum va; ance'^ p. "13). Its e ^ectation and variance- 
iance mat < are 



E(B) = E [Tll)'^ rvj from (66) 



= (X_'l)^ l^E (Y) since X is constant ' 
= (X_"X)"''' X_'j^3 by su^:.sti luti on from 

=B since (rx)""' X^X = I 



and 
(68) 



V (B) - V"[jX^)^)"''^rY fron (66) 



(X^X ^ ® I See Finn, p, 114, ror proof. 
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Using equation (54) we can 51? j see thaT. an unniased (maxim , m 

likelihood) estimate of the errcr variance of Y^, Z, can be obtai"'^:d 

direct !y ■ from the observed data. Let Sr represent the residual 

— t 

err )rs sums of squares and cross prDcucts matrix, and we c ;tain .he 
iance-covariance e^imate by div ding by ^he degrees of freec::m 
s; alar, N-q 



or 



^ Y - XB)" (Y - XB) 
N-q 



Y 'Y - BX^XB 



subst ■ wUting 



by ~mu ti plication and cancelling 



N-q 



a frequently oncaunte: ed .er iivalent 
form 



It should be obvious that each element of I is an estimate of population 

2 

variance and covariance, with a. on the diagonal and a.?, (i f j) 
on the off-diagonal" eel Is . 

Y 

As in the univariate case, the variance in j^^p can be partitioned. 
Let^S^Y represent the total sums of squares and cross products in the 
sample cri terion, scores , Y_' ^^e variation in the predicted 

scores, XT" B^X'XB, called, the sums'^of squares for regression. Then 
the parti tion is ■ 
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\ 



(70) Sy - + S^. 

\ 

\ 

which breaks the total variation into two components: that 
attributable to regression and that attributable^to errc". If 
approximates S_y^ then will be small and the predictor scores 
will predict the criterion scores quite well. If is r:mall 
compared to S^y, then the linear model does not fit the data well . 
and will be quite large. 

The "goodness of fit" of the model can be determined in a way 

that is identical to that in univariate regression. As Press (1972" 

2 

indicates, "In multivariate regression, the value or R i^n be 
computed for each equation seoarately to study the effectiveness of 
each relationship in accounting for observed variation' (j. 195). 

As Timm (1975) indicates, "Estimation theory, us'ipg multi- 
variate linear model, is no different from, employing £ univariate 
models. It is not until hypothesis testing theory is employed that 
the models really differ. Univariate onalycis does not aci^ess 
itself to the dependency that exists amor ^ a set. of £ i^espcmse 
/criterion7 variables" (p. 309). We turn now to the quesT.iOP of 
hypothesis testing. 

Hypothesis Testing for Multivariate Regression 

The logic of hypothesis testing in multivariate multii'e 
regression is similar to that in its univariate counterpart except 
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that multiple criter"on variables are tested simultaneously. The 
purpose of the simultaneous test is to maintain protection levels 
at a point predetermined by the researcher, i.e.,. at a. Were ' 
separate tests run for each dependent variable, the likelihood of 
obtaining a false significant value would ihcreese in direct pro- 
portion to the number of dependent variables being tested, i.e., 
the power of the test decreases. 

The general test for alj^ coefficients of the raw form of the 
model is * 

(71) : B = 0 

which can la examined by using a multivariate analysis of variance 
(MANOVA) table. To conduct the test we must obtain the sums of 
squares and cross products matrices for both hypothesis "and error, 
from which mean squares may be determined. These are, by extension 
from the univariate case 

(72) . = ri - h"^ i 

and \^ i ' ■ .' 

The MANOVA table can then be con^structed 

Insert Table 4 here 
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Normally, of course, we are not interested in testing all of 
the regression equations,' since we seldom test the intercept and 
usually are interested in testing only a subset of the slope 
coefficients. Hence we would partition the model into two sepa^ite 
regressions and develop an hypothesis which would enable us -to test 
the subset. Ttte partitioned model would be : ' 

(73) V =[X, X,^ 

• with the hypothesis that * . . 

(74) . :' B„ = 0 ' ^ . , 

Equatjcns (73 and 74) simply assert that the first set of variables 

(1 = 1. . ..ni) and regressions coefficients ■ Bj (1=0-. . . M) 
are not being tested while the remaining variables X,, (.II = m . . . k) 
and coefficients Bjj (IJ. = m . . . I<^) are under test. It is, of 
course, possible to test the final set of coefficients, B^, by setting 
the ihdicat^r;^rn to k-1. Then we would have Bj equal ^to all k-1 
coefficients and B^j equal to the last set of k' coefficients . 

-There are a number of alternative procedures available to test 
the hypothesis stated in equations (7"i and 74). These all depend on. 
solutions for the roots or eiqenvalues , >^2' * • " ' '^s °^ ' 
characteristic equation 

(75) I \ - I =0 - 



4 1 
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where the roots are ordered from largest, A^, to smallest, X^v 
Anderson (1958, Cp. 8) and Timm (1975, pp. 137-140, 146-149, and 
308-313) describe a number of criteria which may be used to test 
the hypotheses. These tests are provided in Table 5. Tables for 
each of these distributions at selected degrees- of freedom are 
available in Timm (1975) . Since most of these distributions are 
unfamiliar to communication researchers., the table includes tests 
for multivariate hypotheses by approximations to and £ 
statistics whicji have been deriyed.from the other distributions.' 



Table 5 about ' here 



It is instructive to examine the logic of the Wilk's likelihood 
ratio criterion,\ A. It was mentioned earlier that the determinant 
of a variance-covariance matrix can be considered a measure of 
generalized variance. Wilk's A. is the ratio of two generalized 
variances. The numerator contains the variance for the reduced 
model, Q^,' i.e., those variables not under test; the denominator 
contains, the variance for the full set of predictors , i.e., those 
predictors not under test plus those that are being tested,. Thus 



0 ' 



(76) A = . ' ^ 



I Q . i 

It should be apparent that if B^^ = 0 then 0^ = 0 and A = — - — = 1, 
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which is the upper bound of the statistic. On the other hand, to 

the extent that Q. (i.e., X . . . X, ) adds variance over and above 

the reduced model, the denominator will increase and A<1. Thus^ 

the smaller A is, the more that or is adding to criterion 

2 

variation. The £ and ^ tests simply transfor-m the A statistic 

2 

(really, the U distribution) to the more familiar [_ and x. 
distributions. Most computerized MMR routines will print both A 
and either £ or X test statistics. . . 

Testing ttie criterion variables . If A (or any of the otl;ier 
test. criterion) is significant it is then possible to determine * 
which of the criterion variables are being affected by the variabl-es < 
in the predictor set. Univariate ^-ratios may be used to accomplish 
this task. The univariate £ that is largest is the one which shows 
which of the criterion variables is most affected by the predictors,' 
and so on down to the smallest £-ratio which shows the criterion 
variable which is least affected by predictors,. However, as Finn 
(1975, p. 156) points out, these tests are not independent:' "there 
is no necessary relationship of the significance of the univariate 
and multivariate tests for one hypothesis. For example," one or . *' 
more univar ate £'s may be significant and not the multivariate . 
statistic, o\vice versa" (p. 157). Consequently, it is usually 
recommended that in addition to the univariate £ tests, researchers 
examine the simple and multiple correlations to aid in interpretation. 
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^0 deal with this problem of interdependence of multivariate 
Gfid univd'^iate tests for locating effecrs, it is possible to conduct 
a step- uown a naly:sis which will provide independent univariate tests 
for the criterion variables. The term step-down as used here and 
by Bock (1966) and Finn (1975, pp. 157-160) refers- to the fact that 
the criterion variables rather than the predictor variables are 
being examined (eliminated) sequentially. The tests, which are 
described by Roy (1958) require that the researcher impose an a 
priori ordering on the criterion variables. If the researcher has 
no rational basis for such. an ordering, then step - down tests will 
be of little value. The procedures for conducting the tests are 
identical to calculating F tests (for each of the-p criterion, 
variables) with the condition imposed that only the conditional 
variance in the criterion variables be analyzed. The tests proceed 
by regressing all the q predictor variables on y^ , on' y^ eliminating 
on eliminating y^ and y^, etc. The test procedure is a form 
of backward el iinination. , First, the f>^- criterion- variable yp is 
p] -iininated, i.e., the last one in the o.^dered set. Then the next 
'.1-^ the last is eliminated yp -j, etc., down to th^ first criterion, 
y. , At each step an statistic is calculated; if £ is non- 
sinnif leant, and for a particular, criterion cannot be rejected, 
it means that the predictor set does not significantly contribute 
to variation in particular criterion' variable y.. Each time that 
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cannot be rejected, the step -down procedure continues until such 
a time as an mu^t be rejected in favor of the alternative H^. 
At that point testing terminates since all remaining tests are 
nonindependent. See Bock ,(1966) and Finn (1974) for additional 
details on step - down analysis . 

Testing the predictor variables .,,-,^Having determined how the set 
of predictor variables differentially affects the criterion 
variables if is often of interest, as in the univariate case, to 
test hypotheses reqa^ing a subset of the predictor variables. 
Tests for contributions to criterion variance of individual predictors 
ar^^ facilitated by creating ortTiogonal columns of B^, i.e., converting 
the partial regression coefficients to semiparti.al regression 
coefficients. Each semipartial regression coefficient accounts for 
variance in Y not accounted for by predictor variables that preceed 
it in the regression equation! orthogonal ization • process , of 

course^ requires that the researcher impose an a priori order on 
the predictors. When the backward sequential tests on the predictor - 
variables encounter a variable which significantly contributes to 
criterior, variance, the testing procedure must stop, since all . 
subsequent tests are non-independent. It should be apparent from 
this discussion that the 'researcher must rely heavily upon his 
theory to specify the order of the predictor variables so that the 
strongest known predictor is the'first variable in the equation and . 
the weakest is' last. , ' 
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Reestimation 

As in the univariate case, if a researcher either adds or 
deletes variables from the equation with which he began, it is 
important^^to reestima'te the parameters of the resultant or final 
equation. Variance in the non-si gni ficant variables is pooled 
with error variance to make the final estimate. Finn (1975) 
indicates that "Under ideal circumstances, these final estimates 
.should be obtainexl .from a sample other than the one used for 
significance tests" (p. 165). * ' ^ 

Issues and Implications 

In this final section of the paper we shall briefly explore 
several topics which are raised by or related 'to MMR. These should 
help to place the information just presented in a broader context. 
Algori thms for Selecting Predictor Variables 

There are a number of computerized algorithm^ which can be used 
.to select an optimal set of predictors for a regression equation.. 
In fact, Draper and Smith; (1966) identify, six different alternative 
procedures.- Four of these are summarized in this se^ction; for 
details. the reader is referred to the fuller explication in Draper 
and Smith and ai^o to Kerlinger and Pedhazur (1973, pp. 285-295). 

The first procedure is termed all possible regressions. Here 
separate regression equations are calculated for all possible 
combinations of predictor variables : (1) each pred\ctQr variable 
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examined separately, (2) all possible predictor pairs, (3) all 

possible triplets, (4) and in general, all possible k-tuples 

(i = 1 . . . k predictors), where the final k-tuple contains one 

set of all k variables. A moments reflection will indicate that 

a total of 2*^ separate regressions must be determined since each 

variable may either be included or excluded from the equation. 

Within each of the k-tuple regressions (i.e., pairs, triplets^ etc.), 

equations are ordered according to some criterion, usually the 

2 

amount of variance accounted for, R '. The researcher, then selects 
what he considers the best equation, often by looking foW'any 
consistent pattern of variables in the lead,ing equations'' 1n each 
set" (Draper and Smith, p. 162). 

The second procedure is called the backward elimination . There 
are three basic steps. First, calculate a regression equation which 
includes all the k predictor variables. Second, calculate partial 
T-tests for each of the k variables which indicates the significance 
of X, as if it were the last variable entered into the- equation. 

K 

Finally, select the lowest partial F value and compare it with a 
partial-F value set equal "to some predetermined level of significance 
a. If the smallest partial -iF is less than F^, then delete that 
variable and repeat th£ process for k-1 predictor variables. This 
sequence continues until the smallest partial-£ at. any given step 
(k~l predictor variables to 1 predictor) is greater than F^. The 
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variables tha' emain are ccnside- ificant p-^ecictors. 

The thirc clternative for chc sir^i redictors is forwar d 

selection. In ":nis process -.'ariar'es are added to the regression 

equation one at a time rather than deleted from the equation as 

was done in backward elimination. The first step is to select the 

largest zero-order simple correlation coefficient between y and. X ^ . 

This variable is used to construct a simple regression equation. 

Second, an F-test is calculated- to determine whether the simple 

regression of y on the. first predictor variable i s . sign"^f i cant . 

Assum.ing this test is significant, the third step i s ,to calculate 

first-order partial rs between y and all other controlling for 

the predictor variable already in the simple regression equation. 

The X. with the highest partial r (or squared partial r) is then 

selected for the construction of a multiple regression equation wil 

two predictor variables. Fr^iirth, as with the backward elimination 

procedure, partial £-tests ore undertaken to determine whether the 

variable selected for inclusion contributes significantly to the 

variance^ in y over and above the variance accounted for by those 

2 

predictors already in the equation, i.e., R' for the simple 
, regression and for the two variable multiple regression. 
Finally, this process (calculating higher order partial, rs, 
selecting the one of greatest magnitude, creating a new regression 
equation wj.th this variable added, calculating the sinnificance of 
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the increase in variance contributed by this variable) continues 
until the partial-F test indicates that the variable selected for 
inclusion at the next iteration is ■ non-significant. The final 
equation includes all variables found significant up to the point 
at wh.ich adding an additional predictor variable does not 
significantly increase the amount of variance accounted for in the 
criterion variable; all remaining > predictors are considered non- 
significant and omitted' from the equation. 

The fourth procedure, which is called stepwise regression , is 
really a variation on the forward selection procedure. For any 
given set of variables, the magnitude of regression coefficients 
does not vary according to their order in a regression equation. 
For example, the regression coefficients for three variables, call 
them A, B, and C, will be the same regardless of the order in 
which these three variables, are arranged in a 3 variable multiple 
regression equation, i.e., whether variable A is first, second, or. 
last, whether variable B is first, second . . ., etc. If, however, 
a variable, say D, is added to this equation, or one of the three 
variables, A, B, or C, is deleted from the equation, it is highly 
likely that the magnitude of all the coefficients in the new (4 
variable or 2 variable) equation will be considerably altered. The 
reason for this is that regression coefficients are apatterned 
function of a set of variables', such that they account for as much 
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variance as possibl^e^v; i_ or given that p arti cular set of 
predictors and that set only. The aniount ihat the reqression 
,:upf ficients will change is a function of the degree to v;hich they 

variance v/ith each other and the criterion variable(s); the 
j-^ore variance they share in common, the greater the coefficients 
v/i'l change by adding or deleting variables. It should be apparent 
^hat the three proceaures identified so far all require the 
addition or deletior^ of variables. The stepwise procedure atteinpts 
to compensate for the fact that at each step adding a variable 
(remember, it is a variation oti the forward selection procedure) 
could reduce the amount of v:^riation contributed by one or more of 
the variables already in the equation to a point at which it (they) 
v;ould no longer be considered significant. Thus, after' each step 
in which a variable is selected for inclusion in the next larger 
multiple regression equation, a partial-£ test is performed on all 
variables in the equation to which the new variable is being added 
to determine whether each still significantly contributes to 
criterion variance. Any that fail to meet F are d eleted from the 
regression equation and join the set of predictor variables not in 
the equation. Then the process is repeated; new n— order partial 
rs are calculated; the largest is selected for inclusion in a new 
regression equation; a test of the significance of this new 
variable is conducted; if it exceeds F^^ for appropriate v, a 
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separate partial-£ is conducted for each criterion variable already 
in the equation; any- criterion that foils the is removed from 
the equation. The process terminates when there is no new variable 
tha^ can be added to^the equation and significantly increase criterion 
variation. 

An ;al ternati ve procedure, called stagewise regression will not 
be discussed since it is not a least squares procedure. Also, it 
should be noted that there are some "variations of the themes" 
presented in these four alternatives (see Draper and Smith, 1966, . 
pp. 172-173). ' • 

Given the four procedures just described, which one should the 
researcher choose? That, of course, is a judgment call, which each 
person will have to make depending upon his research needs. In 
general, however, all possible regressions is both impractical and 
without recourse to statistical test; it. should be avoided. Draper 
and Smith (p. 172) recommend the stepwise procedure but indicate 
that it can be abused by the inexperienced. In this context, Finn 
(1974), makes an additional important point: 

"Stepwise" procedures which attempt all possible 

prderi ngs , or -search for the best single preijiction 

equation do not' general ly yield valid test statistics, 
, ' and must be interpreted with caution, liith a pre- 

.determined order of predictor variables, val id sequential 

test statistics are obtained. Using a fixed order, it is 
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also possible to test important combinations or sets 

of variables (p. 161 ) . 
Testing the Assumptions 

In this paper we have stressed the importance of the assumptions 
of the model being employed. The only way to dete^nine whether a 
given set of assumptions is viable for any given study is to test 
them. As Timm (1975) indicates, "Plots of residuals against the 
fitted values, against the independent variables, and. sometimes 
against variables not included in the model help to J-^teranne (Ij 
whetner model assumptions are reasonable, (2) the linearity of the 
regression function, and (3) whether important variables have been 
left out of the model" (p. 269). Draper and Smith (1966) review 
these procedures in detail and Daniel and Wood (1971) give several 
illustrative examples; the procedures will be discussed only briefly 
here. 

The logic of the examination is as follows: Recall that our. 
assumptions for the classical normal linear regression model 
specified that the errors were normally distributed with zero mean 
and constant variance and that they were independent. If the • 
.assumptions are correct, the errors or residuals ought to display 
these characteristics. 

The first procedure is to undertake an overall plot of the 
residuals. Normality, or departure from normality, can be judged 
by reference to a table of random normal deviates or by ploting 
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the residuals on standard probability paper. Alternatively, the 
residuals may be trdnsforrned to "unit normal deviate" forFii, in 
which case we would expect that 95v of the residuals would fall 
within the 1.96 a limits. Outliers, which are residuals which lie 
far out in the tails of the distribution, say + 3 or 4 stE' -': 
deviations, can also be identif1e:i, Sc e ... uc.r ... ."egresh 
:c::jjter p-i.k-ges now have subroutines which permit the plotting of 
residuals in unit normal deviate form. 

Another procedure is to plot each residual against the predicted 
value which helped to generate it, i.e., Y^. Here, a horizontal 
band of scores lieing relatively equidistant from zero would indicate 
that the assumptions have been met. If the distribution of rp<;iriii?^ls 
shows divergence (or convergence) -^oss ' es, the . . Mi^Lcnt 

variance assuiiipticn is challenged. If the variance is constant, but 
there is an upward or downward tren*"" ^jio'". ''^f i.-'-re ha-, 

probably been an error 'r vue dftdiys^ j or the constant terr . ''-^"^ 
been omitted. Finally, if the plot looks curvilinear, then the 
assumption of linearity is questioned. .Transformation on the 
variables or extra terms (square or cross-products) may be required 
i n the inodel . 

Residuals may be plotted against the independent variables. 
As with the plot against Y., the existence of a horizontal band of 
residuals is the desired form. Failure to obtain such a plot would 
raise similar sorts of questions as those in the previous paragraph. 



Statistics are available for formally testing residuals. The 
reader is referred to the work by ^nsco.T.be ('361) and Anscombe and 
Tukey (1963) for the details. Other plots are also possible ard 
Draper arr. ST.itr ' vbcoTxr-.eni ttcl .^:]\ residual plot should be 

'rnade that .nakes serse in light of the resea-^ch 'e,g., plr' inc 
res-i duals against c measured on the same subjects but not 

included in the regression equation). For an example of testing 
assumptions in com;r.Mni cation research see Monge, e^ al^- (1976), 
who examined the plot of the residuals for a fully recursive 
structural equation 'POdel of the determinants of ' cormiun ication 
structure in larcc '. ...ions. 

Res,idudls, cf -s jre no". th-i^ ' ,.ay to test assumptions. 
Autoregre- ion can - ascertained by the Durbi n-Watson test. 
Multicol 1 inearity can he determined bv ^y^^i- mq mi 1 of 

'X "X); i f ' i t approe ■ . ^ son:e coUniD of tne matrix is linearly 
dependent upon some other column(s) of the matrix. Homoskedastici ty 
can be examined by" a homogeneity of variance test. Other procedures 
are also available and can be found in most standard econometrics 
texts (e.g., Goldberger, 1964; Johnston, 1972, Kmenta, 1971). The 
important point to remember from this section is that when co^nmunication 
researchers undertake a regression analysis, they should always 
test the assumpti.ons of the model. Only when this becomes standard 
practice in communication research will we be able to have confidence 
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in research findings that employ statistical inference. 
Comparisons with Related Techniques 

KMR has been presented in the context of its unwariate 
counterpart, linear -ultiple regression. To provide a broader 
context it ^ii : be jseful to examine the relationship between MHR 
and several other tecr.?,iques . I will briefly discuss: (1) the 
general 1 inear model , (2) f-IANOVA, (3) canonica'J correlation, and (4) 
general systems of equations. 

MMR has been treated as a special case of the general linear 
model which consists of the functional form of the model and the 
assumptions specified in the Gauss-Markof f theorem (set-up). Many 
of the multivariate techniques, e.g., HANOVA, canonical correlation, 
etc., are derivable from the general linear mode'. To understand 
that point provides two important insights. First, it emphasizes 
that many of the multivariate statistics are highly interrelated 
and not separate, independent techniques. Second, it emphasizes how 
the various techniques differ as a function of the differences in. 
their assuniptions , so that modifying an assumption of the general 
linear model makes it necessary to choose an alternative fo>^m of. 
the model (i.e., a different multivariate technique). 

Comparing \m with MAriOVA helps to illustrate these two 
points. First, Timn (1975) demonstrates how both are special 
cases of the multivariate general lii^ear model. Second, if the 
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researcher assumes categorical rather than ccntinuous rieasurer:ent 

on X- then MANOVA is the aoorcDriate for~ of the nodel to utilize. 
J 

Actually, as Bochner and Fitzpatrick (1977) illustrate in another 
paoer for this conference, a f^AN'OVA model can be analyzed with MMP 
techniques by use of special (durrriy) coding techniques (See also 
Kerlinger and Pedhazur, 1973; Press, 1972; Kmenta-, 1971). 

Canonical correlation is a technique that is appropriate 
when there are two sets of variables, rreasured on ordinal scales. 
The coefficient, is the simple correlation between two random 
variables which are each linear composites of two or more variates. 
Each composite is defined'^oy weights applied to its variates 
■ desjgned to maximize its correlation, R^, with the other composite. 
As Finn (1974) indicates, when one of the "composites" conr^ists of 
only one variate, R^ becomes "the multiple correlation of one 
measure with the other $et. The weights are the partial regression 
coefficients" (p. 188). MMR is. the' appropriate technique when the 
researcher is interested ^'n explaining variation in one set of 
variables by variation in the other; canonical correlation is 
appropriate wnen one is interested in foririnq composites wiiich 
maximally share variance winh eacn other. 

Some econometrics texts treat HHR under the topic of sets of 
linear regression relations (See, e.g., Goldberger, 1964, pp. 201-212). 
This Extension to sets of dependent variables from the univariate 
case is,, as we have shown, quite straightforward. Yet always the 
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emphasis is on explaining variance in the criterion variables by 
variation in the predictor variab'ies. If, however, a researcher is 
interested in test^ing a theory which consists of a system of 
linear relations (either single equations which are par* of a 
larger sys":eTi of equdticns or the entire systen itself), then the 
GLS estirr.tion p^^ocedures described in tnis paper are appropriate 
only in t':e spe'" 'al case of a fully recursive system. Simultaneous 
linear s.trijctur'dl equation systers require alternative estimation 
orocedures which at^e the topic of other papers in this conferer^ce 
(Caopella, 1977; Fink, 1977). 
Coping with Fai ied Assumptions 

In tSiis paper considerable emphasis has been placed on the 
ifiiportance of the consumptions in the classical linear model. Our 
discussion, however, has been limited to OLS techniques; we have 
not examined alternative estimation procedures. Though it is 
beyond the , _ scope of this paper it is important to point out that 
procedures have been developed to permit statistical inference in 
spite o-r' violations of OLS assumptions. Some of these tertini ques , 
such a", two stage least squares to estimate autoregressive 
disturbances and the use of instrumental variables for eri^ors \\\ 
variables (i.e., measurement error in X_) . are simply more 
sophisticated applications of OLS procedures. Others require 
complete abandonment of' OLS in favor of maximum likelihood 
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estimators. Whatever the case may be, the communication researcher 
is strongly encouraged to consult multivariate and econometVic 
texts to discover the appropriate alternatives which are necessitated 
by violations of assumptions in his or her data. 
Advantages a nd Disadvantages of MMR 

Kerlinger aqd Pedhazur (1973) review in depth a number of 
social science studies that employed multiple regression; their" 
conirnents ^give the reader an excellent notion of the breadth of 
'Vr'tl ica tions that are possible with the technique (See Cps. 15 and - 
:*:\/. They also discuss the limitations and strengths of regression 
techniques (pp. 441-445) and that material, some of which has 
already been mentfoned, will not be repeated here. Rather, I • 
'would prefer to make th^ree brief points. 

First, MMR techniques lend themselves to simultaneously 
replicating previous research and developing new findings. This can 
occur by including predictors in the regression equation which .have 
been shown to be important in earlier research. New variables ,may 
then be added and tested for significance over and above those . 
already in the equation. In this way results become cumulative and 
v/e can build comiiiury cati on theories of known predictors. 

The second point is that regression analysis, if done correctly, 
virtually necessitates the formal specification of the regression 
equation. To my vj^ay of thinking this particularly facilitates 
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interpretation of complex relations, particularly interaction terms. 
Furthermore, yi nee traditional ANOVA and more complex MANOVA designs 
may be analyzed via regression techniques, it is possible to develop 
"mixed'' designs which include both categorical and continuous 
pl^edictors. I - ' 

Thirds though we have only discussed cross-sec :1 data in. 
this paper, all econometrics texts also discuss estimation 
procedures fo'r time series data. These may vary from simple lagged 
variables to complex simultaneous structural equation systems. These 
procedures can ^be used to expl icitly capture the time-variant, pro- 
cessional na-ture of many communication phenomena. 
Conclusion ' - / 

At -the outset of this paper we began with a qu^otation from 
Tuk^y (1962) which asserts not only that our most important questions 
are more likely to be vague than- precise, but also that our best 
answers will often be approximate rather than exatt. Asking the . 
right questions is a theoretical endeavor; offer/ng our be^t 
approximations to answers is a statistical' undertaking. 

Having shown in this paper how analysts may expand the relations 
they can examine via . regression techniques to include multiple 
criterion variables, it seems important to stress the necessity for 
•theorists to develop formulations- which will incorporate this 
expanded capability, to develop multivariate theories that can be 
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studied by multivariate multiple regression should lead us a long 
way toward a'sking the right questions and obtaining regression 
coefficient approx-^mations to laws of human communication. 
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Table V. Partition of a data matrix, p into four possibli' ■ ombin.tions of Predictor Variables, 
~ , and Criterion Variables, — , subniatrices, M = P + K, The four cells provide the data partiti 

rlXK ' NXr 

for (l) univariate simple regression, (2);univariate multiple regression, (3) oiultivariate simple ■ 
regression,' and (4) ^yltivariate fiiultipic; "regression. , ' ■ ^ 



'able 2. ANOVA Table for Testing 3 = 0. 
(from Tirm, 1975, p. 273) 



Source 



df 



Total Regression 
Residual 



Total 



k.+ 1 



k -1 



SS 



A A 



E(MS) 



2 



MS,/HS 
n e 
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Table 3.' 



Hypotheses and. Test Statistics for Testing Regression Coefficients 



Hypothesis 
(2) 



(3)' H„.r!) 



or 



Test Stati stic 



CoiimiGnt 



[•', equals zero. Can be used to tost (the 'inkireiil, 



fi . See Johnston, 1972, p. 138. 
0 • 



s 



\]^ eguals some constant, M^. See tata, 19'/1, 
p. 366, 

All coefficients except the intercept are zero. 
See Goldbergef, 1964, p. 176; Timn, 1975, p, 273, 



H 

0 fl 



/ ■ ■ \ 



Some subset of the coefficitjiits -(perhaps the final 
one) is zero. See Goldberger, 1964. 



(5) : 6j = 
(for j / k) 



t = Y 



One regression coefficient equals another. See 
Goldberger, 1964, p, 175. '/ ' 



(6) 



(7) H^: 3-i 
(6 is-"the vector 
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of regression coefficients 
77or a. second equation) 



The sum of two (or more = k) coefficients equals 
a constant.. See tata, 1971,' p. '372. 

Two regression equations are equal. See Kmenta, 
1971, p. 373 for the test statistics. 
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Table 4. MANOVA Table for Testing B_ = 0 
From Tiirm, 1975,, p. 309, ^ 



Source 


df 


SS 


•E(MS) 


} 

1 

Total Regrejiori' 
Residual 


k + 1 
N k - 1 




„ ^ Bl'XB 

L r 


k + 1 


Total 


N 


-: ^ ^'J^i 

I 

1 





I t 



f(' . * ■ 

7,9 



Table 5 

Criteria and Tests for Multivariate Hypotheses 



1. Wilks' likelihood ratio criterion 

' Q I - s 

A = (1 < U''(P, k + 1, N - k - 1) 

jfie ^1 i = l ' 

2. Roy's largest root criterion 

6 - >e" (s, m, n) 

3. Lawley-Hotelling trace criterion (or Hotel 1 ing' s generalized 

2 

T. statistic) 

= ^ = I *.>U« (s,m,n) ■ 

4. Pil lai ' s trace criterion 

s ' , 

= Z .. ^i > V"(s, ni,n) 

i=i 1 + a; 



5. Bartlett's test 



;- X 

6. Fisher's F test 



1= - [(N-1) - -'5 (P + k + if] Log A>Y^^ (pk)- 



F = ^ . 1_^ A f'^ (V V ) 
V, A- 

where • , 

s = niin (v^, p) = (k+1 , p) 
lU - V, I -1 



V - U - 1 
n = e 



2 

= deqrees of freedom 
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